Spider Friendly Content Pages
by Bob McElwain


It is virtually impossible to build a site in which each page
brings good search engine position. The home page, for example,
will likely change frequently. Thus spiders will not find it the
same when they return, which they do, roughly once each month.

Pages devoted to selling product do not often rank well.  
The same is true of a page where visitors can subscribe to your
newsletter. Or the one you pop up to say thanks when they do
subscribe. So how does one go about getting good search engine
positions?

Great Content Is The Answer

So what is great content? Any information surfers may need. 
However, it must also be a topic that enhances your site purpose. 
That is, there is no room on a site devoted to baseball for a
piece describing the inner workings of steam engines.

Assuming you have a clear read on who your visitors are, then
it's only a matter of selecting a topic likely to be of interest
to at least some of them. Given this, write the page for your
visitors, not the search engines. Then do what you can to make
the spiders happy.

Happy Spiders?

Not likely. It is impossible to please them all. Some see
"Market," "MARKET," and "market" as separate words; others see
only one repeated three times. Some see "market" as "marketing;"
most require a specific match. "Markets" may be seen as
"Market," but in other cases both forms may be required.

Okay, we'll include all cases in our keyword tag: Market,
MARKET, market, Markets, MARKETS, markets, Marketing, MARKETING,
marketing.

That's got it covered fine, but how do we make this work with
a spider that considers more than three repetitions as spam? One
that might even consider all of the above as 9 repetitions of one
word?

You Can't Get There From Here

Search engines are competing in a multi-billion dollar race. 
The winner will be the one that can most consistently present 
the most relevant information available in response to a query.

Be assured that with the stakes this high, the competition 
is fierce. They are not about to reveal their latest wrinkle to
improve their listings. Which leaves us with empirical evidence
and educated guesses.

Try to sort this all out for each search engine, and you'll
go crazy.  Not to mention constant changes which mean one or more
of the carefully defined "rules" no longer holds. 

Even supposing you had an accurate listing of the rules for
each engine. Would you seriously consider creating a separate
page for each? Not me. I have much more profitable ways in
which to use my time.

Take the longer view. Spiders are getting smarter every day. 
And they are becoming smarter at a rapidly increasing rate. Some
are now reading a page as if with a thesaurus in hand, thus being
able to see house and home as having similar meanings.

Grammar checkers exist; I expect to see these and related
tools implemented in spider logic. In the not-to-distant future,
those keyword-rich doorway pages are going to be discarded. 

Meanwhile we need to create some great content pages and try
to make the spiders as happy as possible. Here's my approach.

Finding Keywords

Given a topic and a mental draft of what needs to be written,
I identify 1 to 3 keyword phases (I don't think individual
words work well now). I work at this, trying to put myself in
the shoes of one who will search for this information. If I am
building a major page, or one of a set of related topics, I may
take the time to visit Overture to find phrases actually entered.
(For details, mailto:keywordlot@sitetipsandtricks.com)

Meta Tags

I build a rough draft of the title and description tags
before beginning to write. They must serve two purposes. First,
the title is the headline of an ad which draws the reader into
the ad copy (description). And the description must compel a
click to my site. Second, to please the spiders, keywords need 
to be included, and the closer to the beginning of the statements 
the better (I try not to think about the fact that some spiders 
will ignore both tags).

Since Excite limits a title to 70 characters, I try to hold
under this. If I go over, I try to work things out so that
truncation does little harm. I try to hold the description under
150 characters, the limit at AltaVista. I use these limits
because together, AltaVista and Excite dominate among search
engines.

These two tags are so vitally important, that I review them
as often as I write. 

The keyword tag, on the other hand, gets little attention. 
This tag has been so abused, I simply can't get a handle on what
works best. Some meta tag checkers still claim you ought to use
all 1000 characters allowed. This seems unwise.

I include only my keyword phrases, all in lower case. But 
I do add the plural case and "ing" when appropriate.

The Content

When I begin to write, I think only of communicating as
effectively as possible with my visitor. I keep the keywords in
mind and seek to build in a theme based upon them. After editing
a first draft, I will often lay it aside for a day or two before
continuing. My visitors are my target here, not the spiders.

The Spider's Turn

If I can build some header tags with keywords, I will. I don't 
bother with ALT assignments or comments in the source, although 
this reportedly gives a boost with some search engines.

I work at including keywords as close to the top of the page
as possible, in the first 100-200 words. For this is the part 
of the page in which one expects to find the subject defined,
followed by further explanation and expansion. Even now, 
spiders also expect this.

I also work at rephrasing things to add more repetitions 
of keywords and to bring them as close to the beginning of
paragraphs as possible. 

And I make a point of repeating the keywords in the close 
of the page, a sort of "theme" wrap up, if you will.

One further thing I do is look for words I used so frequently 
they may dilute the weight of the keywords. For example, if I 
have used "buildings" too often, I may replace some instances 
with "structures" or a specific name for a type.
   
But throughout, I absolutely refuse to sacrifice readability. 
To me, my visitor is far more important than any search engine.

Other Guidelines

Keyword density is the percentage of words that the keywords
are to the total number of words. It is considered quite
differently by different spiders. Some suggest as much as 15% 
of a page be keywords. To me this is nonsense, for it makes the
page unintelligible to a visitor. I have never been able to get
above 2% without decreasing readability, even when using three
keywords.
   
Page length expected also differs drastically. Many claim
short pages are better. 300-600 words is often suggested. But
Excite doesn't care how long a page is. I say what needs saying
as briefly as possible and call it good.

Never Look Back

When the page is polished, I submit it to the major search
engines. Then I do something you really ought to try.

I never look back. The page is up and that's that. I've 
got more important things to do than worry about what position
it has today. Or where it may be tomorrow. If I've done the 
job properly, my visitors will enjoy the page. And that's the 
end of it.

================================================================
Bob McElwain has been involved in Web marketing and consulting 
since 1993. Want to build a winning site? Improve one you already 
have? Fix one that's busted? Get ANSWERS. Subscribe to "STAT 
News" now! mailto:join-stat@lists.dundee.net
Site:   Phone: 209-742-6783
================================================================